Vehicle accessibility determination device

ABSTRACT

An image convertor converts an original image captured by a front camera. The original image includes a road surface around a vehicle. A three-dimensional object detector detects from a virtual image a three-dimensional object having a height from the road surface. A vehicle accessibility determiner determines whether the vehicle can access to the inside of the detected three-dimensional object or to a clearance among other three-dimensional objects.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is based on and claims priority from JapanesePatent Application No. 2015-024261, filed on Feb. 10, 2015, thedisclosure of which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

This invention relates to a device for processing images around avehicle that are captured by a camera attached to the vehicle, andparticularly to processing by converting the images around the vehicleinto an overhead image.

BACKGROUND ART

When coordinate conversion is performed on images around a vehicle intooverhead images or bird's-eye view images to display the convertedimages, three-dimensional objects are displayed with distortion.Therefore, it is difficult to determine how close the vehicle can accessto the three-dimensional objects by seeing the overhead images.Accordingly, a device that corrects the distortion of three-dimensionalobjects generated by image conversion into overhead images, and displayscorrected images has been proposed, for example (Patent Literature 1: JP2012-147149 A, for example).

SUMMARY Technical Problem

A vehicle periphery image display device disclosed in Patent Literature1 can correct the distortion of the three-dimensional objects which areplaced on a road surface. However, in a case where a portion of thethree-dimensional objects is not placed on the road surface but floatingin the air, when the floating portion is converted into overhead images,the portion is projected to be deformed and inclined toward the roadsurface in a direction away from the vehicle. Therefore, relying on theoverhead images, the vehicle may hit the three-dimensional objects whenthe vehicle is moved close to the three-dimensional objects. Inaddition, when the three-dimensional object is an object such as agarage the vehicle enters, it cannot be determined whether the vehiclecan enter the garage or not only by seeing the overhead images.

The present invention is made in view of above problems. The presentinvention determines whether there is a space to which a vehicle canaccess in a three-dimensional object detected on a road surface aroundthe vehicle, and determines whether the vehicle can access to the spaceor not.

Solution to Problem

To solve the above problems, a vehicle accessibility determinationdevice of the present invention includes an imager that is attached to avehicle and captures a range including a road surface around thevehicle; an image convertor that converts an original image captured bythe imager into a virtual image viewed from a predetermined viewpoint; athree-dimensional object detector that detects from the virtual image athree-dimensional object having a height from the road surface; and avehicle accessibility determiner that determines whether the vehicle iscapable of accessing to an inside of the three-dimensional object or toa clearance of other three-dimensional objects.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a functional block diagram illustrating a functionalconfiguration of a vehicle accessibility determination device accordingto a first embodiment of the present invention.

FIG. 2 shows a mounting condition of an imager to a vehicle in thevehicle accessibility determination device according to the firstembodiment.

FIG. 3 is a functional block diagram illustrating a detailed functionalconfiguration of a three-dimensional object detector in the vehicleaccessibility determination device according to the first embodiment.

FIG. 4 is a hardware block diagram illustrating a hardware configurationof the vehicle accessibility determination device according to the firstembodiment.

FIG. 5 is a flowchart illustrating a flow of a series of processesperformed in the first embodiment.

FIG. 6A shows an example of an original image captured by a frontcamera.

FIG. 6B shows an example of a virtual image (overhead image) convertedfrom the image of FIG. 6A and viewed from directly above.

FIG. 7A is a view illustrating the operation of a three-dimensionalobject detection process, and shows an example of a virtual image(overhead image) converted from an original image captured at time t−Δt.

FIG. 7B is a view illustrating the operation of the three-dimensionalobject detection process, and shows an example of a virtual image(overhead image) converted from an original image captured at time t.

FIG. 7C is a view illustrating an example of performing a framedifference (first frame difference) for subtracting a virtual image attime t−Δt from a virtual image at time t.

FIG. 7D is a view illustrating an example of performing a framedifference (second frame difference) for subtracting a virtual image attime t expected based on the virtual image at time t−Δt from the virtualimage at time t.

FIG. 7E shows an example of a result of detecting a three-dimensionalobject region.

FIG. 8 is a flowchart illustrating a flow of a three-dimensional objectdetection process.

FIG. 9A is a view illustrating the operation of a three-dimensionalobject area extraction process, in which an area representing thedetected three-dimensional object by the three-dimensional object areaextraction process is superimposed on the original image.

FIG. 9B is a view illustrating windows set on the original image toevaluate a density distribution and to identify the three-dimensionalobject area.

FIG. 9C is a graph illustrating an example of a density histogram ineach of the windows shown in FIG. 9B.

FIG. 10 is a view illustrating another example of extracting athree-dimensional object region.

FIG. 11 is a flowchart illustrating a flow of a three-dimensional objectarea extraction process.

FIG. 12A is a first view illustrating the operation of a vehicleaccessibility determination process, and a process for calculating aroad surface projecting position, in which the three-dimensional objectis projected to the road surface from directly above.

FIG. 12B is a second view illustrating the process for calculating theroad surface projecting position, in which the three-dimensional objectis projected to the road surface from directly above.

FIG. 12C is a view illustrating an example of a detection result of aroad surface grounding line N representing the road surface projectingposition of the three-dimensional object.

FIG. 13 is a flowchart illustrating a flow of a vehicle accessibilitydetermination process.

FIG. 14A is a first diagram showing a display example of a vehicleaccessibility range.

FIG. 14B is a second diagram showing a display example of the vehicleaccessibility range.

DESCRIPTION OF EMBODIMENTS

Embodiments of a vehicle accessibility determination device of thepresent invention are described with reference to the drawings.

First Embodiment

In this embodiment, the present invention is adopted to a vehicleaccessibility determination device that detects a three-dimensionalobject around a vehicle, determines whether the vehicle can access tothe detected three-dimensional object, and then informs a driver of aresult.

(Description of Overall Configuration)

First, the functional configuration of this embodiment is described withreference to FIGS. 1-3.

As shown in FIG. 1, a vehicle accessibility determination device 100 ofthe present invention includes an imager 12, an image inputting portion20, an image convertor 30, a three-dimensional object detector 40, avehicle accessibility determiner 50, and a vehicle accessible andinaccessible range display 60. The imager 12 is mounted to a vehicle 10to capture images including a road surface in front of the vehicle 10.The image inputting portion 20 converts an image signal output from theimager 12 into an original image 70 which is in a digital image formatcapable of being processed by a computer. The image convertor 30converts the original image 70 into a virtual image 72 viewed from apredetermined viewpoint. The three-dimensional object detector 40detects from the virtual image 72 a three-dimensional object having aheight from the road surface. The vehicle accessibility determiner 50determines whether the vehicle 10 can access to the inside of thedetected three-dimensional object or to a space defined between thedetected three-dimensional objects. The vehicle accessible andinaccessible range display 60 displays a range to which the vehicle canaccess or a range to which the vehicle cannot access as a result of thevehicle accessibility determination by the vehicle accessibilitydeterminer 50.

As shown in FIG. 2, the imager 12 is attached to the front portion ofthe vehicle 10 and captures the inside of a front observation range 14in the visual field range of 180 degrees. The front observation range 14includes a road surface right before the vehicle 10.

As shown in FIG. 3, the three-dimensional object detector 40specifically includes a first frame difference calculator 40 a, a secondframe difference calculator 40 b, an edge detector 40 c, and athree-dimensional object area clustering portion 40 d. The first framedifference calculator 40 a performs a frame difference between twovirtual images 72(t), 72(t−Δt) that are generated from two originalimages 70(t), 70(t−Δt), respectively. The two original images 70(t),70(t−Δt) are captured by the imager 12 (FIG. 1) at a predetermined timeinterval, converted by the image convertor 30 (FIG. 1), and obtained atdifferent times. The second frame difference calculator 40 b generatesan expected virtual image 72′(t) at time t from the virtual image72(t−Δt) generated from the original image 70(t−Δt) obtained at time(t−Δt). The second frame difference calculator 40 b then performs aframe difference for subtracting the expected virtual image 72′(t) fromthe virtual image 72(t) actually obtained at time t. The edge detector40 c detects from the virtual image 72(t) pixels which change inbrightness more than adjacent pixels. In other words, the edge detector40 c detects pixels constituting an edge. The three-dimensional objectarea clustering portion 40 d detects an area which is consideredconstituting a three-dimensional object based on the calculated resultof the first frame difference calculator 40 a and the calculated resultof the second frame difference calculator 40 b, and the detected resultof the edge detector 40 c.

As shown in FIG. 1, the vehicle accessibility determiner 50 specificallyincludes a three-dimensional object area extracting portion 50 a, a roadsurface projecting position calculating portion 50 b, and a vehicleaccessible space identifying portion 50 c. The three-dimensional objectarea extracting portion 50 a extracts an area corresponding to thethree-dimensional object detected by the three-dimensional objectdetector 40 from the original image 70. The road surface projectingposition calculating portion 50 b calculates a road surface projectionposition. The road surface projection position indicates a limitposition in which the vehicle can access to the three-dimensional objectrelative to the three-dimensional object area extracted by thethree-dimensional object area extracting portion 50 a. The vehicleaccessible space identifying portion 50 c identifies whether there is aspace inside the three-dimensional object or in a clearance among otherthree-dimensional objects to which the vehicle 10 can access.

Now, the configuration of hardware is described with reference to FIG.4. The vehicle accessibility determination device 100 according to thisembodiment includes ECU (electronic control unit) 110, a front camera 12a, a vehicle condition sensor 140, and a monitor 150. ECU 110 is mountedto the vehicle 10 and performs required image processing and/orarithmetic processing. The front camera 12 a is connected to ECU 110 andconstitutes the imager 12 (FIG. 1). The vehicle condition sensor 140calculates a moving direction and a moving amount of the vehicle 10 bysensing the behavior of the vehicle 10. The vehicle condition sensor 140consists of a steering angle sensor and/or a distance sensor. Themonitor 150 displays a processed result of the vehicle accessible andinaccessible range display 60 (FIG. 1).

ECU 110 includes CPU 112, a camera interface 114, a sensor interface116, an image processing module 118, a memory 120, and a displaycontroller 122. CPU 112 receives and transmits required data, andexecutes programs. The camera interface 114 is connected to CPU 112 andcontrols the front camera 12 a. The sensor interface 116 obtainsmeasured results of the vehicle condition sensor 140. The imageprocessing module 118 performs image processing with predeterminedprograms stored in the module 118. The memory 120 stores intermediateresults of the image processing, required constants, programs, or thelike. The display controller 122 controls the monitor 150.

The image inputting portion 20, the image convertor 30, thethree-dimensional object detector 40, the vehicle accessibilitydeterminer 50, and the vehicle accessible and inaccessible range display60 described in FIG. 1 are respectively controlled with pieces ofsoftware, each of which achieves an operation described below. Thepieces of software are stored in the memory 120 and executed whenneeded. Note that the pieces of software may be stored in CPU 112 and/orthe image processing module 118 if required.

(Description of a Flow of Processes Performed in the VehicleAccessibility Determination Device)

Now, a series of the processes in the vehicle accessibilitydetermination device 100 is described with reference to a flowchartshown in FIG. 5. Note that each of the processes is now describedbriefly but will be described in detail later.

(Step S10)

An image conversion process is performed. Specifically, the capturedoriginal image is converted into the virtual image.

(Step S20)

A three-dimensional object detection process is performed. The detailedprocess will be described later.

(Step S30)

A three-dimensional object area extraction process is performed. Thedetailed process will be described later.

(Step S40)

A vehicle accessibility determination process is performed. The detailedprocess will be described later.

(Step S50)

A vehicle inaccessible range display process is performed. The detailedprocess will be described later.

Hereinafter, each of the processes performed in the vehicleaccessibility determination device 100 is described in order.

(Description of Image Conversion Process)

First, the operation of the image conversion process is described withreference to FIG. 1 and FIGS. 6A, 6B. The image conversion process isperformed in the imager 12, the image inputting portion 20, and theimage convertor 30 shown in FIG. 1.

Specifically, the image inputting portion 20 converts an output from thefront camera 12 a (FIG. 4), which constitutes the imager 12, into adigital image. The converted digital image is input to the imageconvertor 30 as the original image 70 shown in FIG. 6A. Note that theoriginal image captured at time t is indicated with 70(t).

The original image 70(t) in FIG. 6A is a captured image of a roadsurface 80 in front of the vehicle 10 (FIG. 1). A lane marker 81 isdrawn on the road surface, and a garage 82 (a three-dimensional object)for parking the vehicle 10 is located behind the lane marker 81. Thegarage 82 has leg portions 83, 85 on the left and the right sides of thegarage 82, respectively. Note that a vehicle shadow 87 which is theshadow of the vehicle 10 is shown in the bottom of the original image70(t).

The image convertor 30 converts the original image 70(t) shown in FIG.6A into the virtual image 72(t) (overhead image or bird's-eye viewimage) viewed from directly above the vehicle 10 as shown in FIG. 6B.Though the conversion method is not described in detail, the virtualimage 72(t) is obtained by performing the coordinate conversion on theoriginal image 70(t) using installation layout information (a height ofthe camera, a depression angle of the camera, lens parameters) of thefront camera 12 a (FIG. 4). The coordinate conversion is to be projectedto the road surface on which the vehicle 10 is located. Note that thevirtual image converted from the original image 70(t) is indicated with72(t).

During the generation of the virtual image 72(t), in the garage 82 (thethree-dimensional object) shown on the original image 70(t) captured bythe front camera 12 a, the left and right leg portions 83, 85 areprojected as deformed on the virtual image 72(t). Specifically, the leftand right leg portions 83, 85 are projected to be inclined toward theroad surface in a direction away from the vehicle 10 as the leg portions83, 85 go upward. In other words, the leg portions 83, 85 are projectedto be deformed such that a width between the leg portions 83, 85 becomeswider in the top of the virtual image 72(t). The deformation of the legportions 83, 85, that is, the deformation in areas each having a heightfrom the road surface occurs to spread radially toward the periphery ofthe virtual image 72(t) from a installation position P1 (FIG. 6B) of thefront camera 12 a (FIG. 4).

Further, invisible areas 86, 86, which are outside the field of view ofthe front camera 12 a, are generated in the virtual image 72(t).Accordingly, predetermined gray values (0, for example) are stored inthe invisible areas 86, 86.

(Outline Description of the Three-Dimensional Object Detection Process)

Next, with reference to FIG. 3 and FIGS. 7A to 7E, the operation of thethree-dimensional object detection process is described. Thethree-dimensional object detection process is performed in thethree-dimensional object detector 40 in FIG. 3.

As shown in FIGS. 7A, 7B, the three-dimensional object detector 40detects three-dimensional objects from the virtual images respectivelygenerated from two original images that are captured at a time intervalΔt. In this embodiment, the description is made as the three-dimensionalobjects are to be detected from the virtual image 72(t) generated fromthe original image 70(t) captured at time t, and the virtual image72(t−Δt) generated from the original image 70(t−Δt) captured at timet−Δt.

First, as shown in FIG. 7C, a frame difference of the two virtual images72(t), 72(t−Δt) is calculated. This process is performed in the firstframe difference calculator 40 a shown in FIG. 3.

Then, as shown in FIG. 7D, the virtual image 72(t) at time t is expectedfrom the virtual image 72(t−Δt) to generate an expected virtual image72′(t). A frame difference of the expected virtual image 72′(t) and thevirtual images 72(t) actually obtained at time t is calculated. Thisprocess is performed in the second frame difference calculator 40 bshown in FIG. 3.

Specifically, the expected virtual image 72′(t) is generated as follows.The vehicle condition sensor 140 (FIG. 4) measures the traveling amountand the moving direction of the vehicle 10 for the time interval Δtwhenever necessary. Then, the virtual image 72(t−Δt) is translated androtated to correspond to the measured traveling amount and movingdirection for the time interval Δt, and accordingly, the expectedvirtual image 72′(t) at time t is generated. At this time, the expectedvirtual image 72′(t) is generated on the assumption that the whole roadsurface is reflected on the virtual image 72(t−Δt).

Note that the frame difference in the second frame difference calculator40 b may be performed between the expected virtual image 72′(t−Δt) andthe virtual image 72(t−Δt) actually obtained at time t−Δt, aftergenerating the expected virtual image 72′(t−Δt) at time t−Δt based onthe virtual image 72(t) at time t.

Performing the frame deference between the virtual image 72(t) and theexpected virtual image 72′(t) matches positions of patterns drawn on theroad surface such as the lane marker 81 shown in FIG. 7B. Accordingly,the lane marker 81 can be restrained or deleted as shown in FIG. 7D. Onthe other hand, the vehicle shadow 87 on the road surface is formed inthe substantially same positions even when the vehicle 10 moves during ashort time interval for which the frame difference is performed. As aresult, the vehicle shadow 87 cannot be deleted with the framedifference between the virtual image and the expected virtual image asshown in FIG. 7D.

As opposed to above, the vehicle shadow 87 formed in the substantiallysame positions can be deleted as shown in FIG. 7C with the framedifference of the two virtual image 72(t), 72(t−Δt) actually obtained attwo different times. Patterns drawn on the road surface such as the lanemarker 81 cannot be deleted with this frame difference since anobservation position moves as the vehicle moves.

Here, a result of the frame difference performed in the first framedifference calculator 40 a (FIG. 7C) is compared with a result of theframe difference performed in the second frame difference calculator 40b (FIG. 7D).

First, when areas having substantially same features (shapes, forexample) are not detected with the frame difference in the first framedifference calculator 40 a in vicinity to areas detected as a result ofthe frame difference in the second frame difference calculator 40 b, theareas detected with the frame difference in the second frame differencecalculator 40 b can be presumed as the vehicle shadow 87 on the roadsurface or reflections that occur on the road surface by the sun or thelighting lamp. Then, the areas are deleted since the areas aredetermined not to represent three-dimensional objects.

Next, in remaining areas after the above determination, only areasinclined toward the road surface in the direction away from the vehicle10 are detected as three-dimensional objects each having a height fromthe road surface. Specifically, in the example of FIGS. 7C, 7D, only theleg portions 83, 85 are detected.

The fact that the areas obtained as a result of the frame differencesare inclined toward the road surface in the direction away from thevehicle 10 can be determined by referring to a result of an edgedetection from the vertical image 72(t) and by confirming that edgedirections in the areas obtained as a result of the frame differencesextend along radial lines through the installation position P1 (FIG. 6B)of the front camera 12 a (FIG. 4).

(Detailed Description of Three-Dimensional Object Detection Process)

Next, the three-dimensional object detection process is specificallydescribed with reference to FIGS. 7C to 7E.

First, a result of the first frame difference shown in FIG. 7C isbinarized with a predetermined threshold value, and pixels having grayvalues larger than the threshold value are detected as firstthree-dimensional object candidate areas (not shown). This processdetects areas such as the leg portions 83, 85 and/or the lane marker 81.

Next, a result of the second frame difference shown in FIG. 7D isbinarized with a predetermined threshold value, and pixels having grayvalues larger than the threshold value are detected as secondthree-dimensional object candidate areas (not shown). This processdetects areas such as the leg portions 83, 85 and/or the vehicle shadow87.

Only areas having same shapes (features) and located in close positionsrelative to the detected first and second three-dimensional objectcandidate areas are selected. That is, so called deletion of nonthree-dimensional objects is performed. This process can delete nonthree-dimensional objects whose positions are moved as the vehiclemoves. Specifically, the areas of the lane marker 81 and/or the vehicleshadow 87 can be deleted. The deletion of the non three-dimensionalobjects can be carried out, for example by performing a logical ANDoperation on the detected first and second three-dimensional objectcandidate areas.

Note that features used in this embodiment are not limited to shapes ofthe areas. Specifically, areas located in close positions relative toeach other may be detected using the luminance differences of the areas.Further, as features, similarity (similarity in edge directions, edgeintensity) obtained as a result of performing the edge detection on thevirtual images may be used. Also, similarity of a histogram of the edgedetection result or a density histogram obtained from each of aplurality of small blocks made by dividing the virtual images, or thelike may be used.

Next, the edge detector 40 c (FIG. 3) performs an edge detection on thevertical image 72(t). The edge detection is performed by calculatingdifference in brightness between adjacent pixels as generally performed.

Relative to remaining areas after the deletion of the nonthree-dimensional objects, a direction in which each of the areasextends is evaluated by referring to a result of the edge detection ofthe pixels in the same positions as the remaining areas. The areasconstituting the three-dimensional objects are converted to areasradially extending from the installation position P1 (FIG. 6B) of thefront camera 12 a (FIG. 4) toward the periphery of the virtual image72(t). Accordingly, the areas constituting the three-dimensional objectsare determined by confirming the shapes of the areas and/or theextending directions to meet the above condition.

At this time, the areas detected with the frame differences are notdirectly determined as the three-dimensional areas, but the edgedetection result of the virtual image 72(t) is referred. Accordingly,the influence of the time variation of the exposure characteristic ofthe camera, shadows, lighting, which may be mixed in the results of theframe differences, can be reduced. In addition, since the edge detectionresult of the virtual image 72(t) is referred, the three-dimensionalobject area at time t−Δt may not be left as afterimages, and anerroneous detection is suppressed in the case where thethree-dimensional objects move. Accordingly, the detection performanceof the three-dimensional objects can be further improved.

The three-dimensional object areas 90, 92, which are considered toconstitute three-dimensional objects, are detected with a series of theprocesses. Then, the lowermost edge positions of the detectedthree-dimensional object areas 90, 92 are detected as road surfacegrounding positions 90 a, 92 a. The road surface grounding positions 90a, 92 a represent positions where the three-dimensional objects are incontact with the road surface.

(Description of a Flow of Three-Dimensional Object Detection Process)

Next, a flow of the three-dimensional object detection process isdescribed with reference to a flowchart of FIG. 8.

(Step S100)

The frame difference is performed in the first frame differencecalculator 40 a.

(Step S110)

The frame difference is performed in the second frame differencecalculator 40 b.

(Step S120)

A result of Step S100 is compared with a result of Step S110, and areasmoving with the movement of the vehicle 10 are deleted as the nonthree-dimensional objects.

(Step S130)

The edge detection is performed on the virtual image 72(t).

(Step S140)

In the remaining areas as a result of Step S120, only areas inclinedtoward the road surface in the direction away from the vehicle 10 aredetected.

(Step S150)

Road surface grounding positions of the detected areas are detected.

(Description of Three-Dimensional Object Area Extraction Process)

Next, relative to the three-dimensional objects detected from thevirtual image 72 (overhead image), areas corresponding to the detectedthree-dimensional objects are extracted from the original image 70.Hereinafter, the operation of the three-dimensional object areaextraction process is described with reference to FIGS. 9A to 9C.

Contrary to the case where the virtual image 72 is created, a reverseoverhead view conversion is performed on the three-dimensional objectareas detected from the virtual image 72 to convert the areas into thecoordinate system of the original image 70. The reverse overhead viewconversion identifies the three-dimensional object areas in the originalimage 70(t) as shown in FIG. 9A. However, in the case of FIG. 9A, upperportions of the garage 82 (three-dimensional object) are framed out whenconverted into the virtual image. Accordingly, only thethree-dimensional object areas 90, 92, which are lower portions close tothe ground surface, are extracted and superimposed. Since the gray valuestored in the original image 70 is used in a subsequent process, thesuperimposition in this embodiment means to superimpose another layer onthe original image 70.

Next, as shown in FIG. 9B, a rectangular area W1 is set in the originalimage 70(t). The outer edge of the rectangular area W1 contacts theouter edges of the three-dimensional object areas 90, 92 in a lateraldirection. The vertical size of the rectangular area W1 is set to apreset predetermined value. Then, the three-dimensional object areaextracting portion 50 a creates a density histogram H (W1) in the areaof the original image 70(t) corresponding to the rectangular area W1. Anexample of the density histogram H (W1) created as above is shown inFIG. 9C.

As can be seen from FIG. 9C, the density histogram H (W1) includes twoareas, namely a dark area and a bright area. The dark area formsthree-dimensional object areas 90, 92 and the bright area formsnon-three-dimensional object areas.

Then, rectangular areas W2, W3, and W4 are set by increasing thevertical size of the rectangular area W1 by a predetermined value, andthe density histograms H (W2), H (W3), and H (W4) of the original image70(t) corresponding to each of the rectangular areas are created eachtime the rectangular area is set. Examples of the density histograms H(W2), H (W3), and H (W4) created as above are shown in FIG. 9C.

As can be seen from FIG. 9C, when the rectangular areas and thethree-dimensional object areas 90, 92 overlap, the density histogramshave similar forms. Specifically, in the example of FIG. 9C, from thedensity histograms H (W1), H (W2), and H (W3) obtained when therectangular areas W1, W2, and W3 are set, it can be noticed that twoareas, that is the dark area forming the three-dimensional object areas90, 92 and the bright area forming the non-three-dimensional objectareas appear. Then, with regard to the density histogram H (W4) obtainedwhen the rectangular area W4 completely overlaps the three-dimensionalobject areas 90, 92, the similarity of the density distribution to theothers is disrupted.

In this embodiment, as shown in FIG. 9C, a plurality of rectangularareas Wi (i=1, 2, . . . ), each having a different vertical size, areset; the density histograms H (Wi) in the rectangular areas Wi arecreated; similarities of the created density histograms H (Wi) areevaluated; and areas corresponding to the three-dimensional objects areextracted from the original image 70(t).

Note that as a measure for evaluating similarities in the densityhistograms H (Wi), various methods such as a Euclidean distancedetermination method, a histogram intersection method, or the like havebeen proposed and may be used to evaluate the similarities. However,since the sizes of the rectangular areas Wi (i=1, 2, . . . ) to be setare different from each other, an area (dimension) of each of thecreated density histograms H (Wi) differs from each other. Therefore, inorder to calculate the similarities, it is necessary to perform anormalization process in advance so that the areas (dimensions) of thedensity histograms H (Wi) become equal. The similarities in the densityhistograms H (Wi) may be evaluated based on a mode method assumingbimodality of the histograms, stability of binarization threshold bydiscriminant analysis, or the like, by considering variations in thearea (dimension) of the three-dimensional object areas and thenon-three-dimensional object areas in the rectangular areas Wi.

In addition, the rectangular areas Wi (i=1, 2, . . . ) contacting thethree-dimensional object areas 90, 92 (FIG. 9A) are set in the exampleof FIG. 9B. This is because it is assumed that there are floating areasin the air between the three-dimensional object areas 90 and 92.

Therefore, it is possible to set areas contacting the three-dimensionalobject areas 90, 92, respectively, and to obtain the twothree-dimensional object areas. However, in that case, after extractingthe three-dimensional object areas respectively, it is determinedwhether these three-dimensional object areas are one mass or not. Thisdetermination is made by, for example, the similarity evaluation of thedensity histograms as the extraction of the three-dimensional objectareas described above. If the areas are one mass, it is necessary to beintegrated as one area.

Next, another example of extracting three-dimensional object areas isdescribed with reference to FIG. 10. FIG. 10 is an original image 70(t)captured at time t and showing a condition in which another vehicle 11(three-dimensional object) is parked in front of the vehicle 10. FIG. 10also shows a condition in which a three-dimensional object area 94detected by the three-dimensional object area detection is superimposedto the original image 70(t). Locations on which tires of vehicle 11(three-dimensional object) are grounded are detected as road surfacegrounding positions 94 a, 94 b.

The three-dimensional object area extraction process described above isperformed on the original image 70(t) shown in FIG. 10. That is, therectangular areas Wi (i=1, . . . , N) contacting the three-dimensionalobject area 94 are set, and similarities of the density histograms H(Wi) are evaluated. Thereby, the rectangular area Wn contacting vehicle11 can be extracted.

(Description of a Flow of Three-Dimensional Object Area ExtractionProcess)

Next, a flow of the three-dimensional object area extraction process isdescribed with reference to a flowchart of FIG. 11.

(Step S200)

The three-dimensional object areas detected from the virtual image 72are inversely converted and superimposed on the corresponding positionsin the original image 70.

(Step S210)

A plurality of rectangular areas Wi (i=1, 2, . . . ) are set asthree-dimensional object candidate areas. The rectangular areas Wicontact in a lateral direction the three-dimensional object areassuperimposed on the original image 70.

(Step S220)

The density histograms H (Wi) are created with regard to areascorresponding to the rectangular areas Wi from the original image 70.

(Step S230)

Similarities of the created density histograms H (Wi) are calculated tofind a set of the density histograms H (Wi) having a high degree ofsimilarity. Then, in the set of the density histograms H (Wi) determinedto have the high degree of similarity, a rectangular area Wi having themaximum vertical size is set as the three-dimensional object area.

(Step S240)

It is determined whether the three-dimensional object area is set ornot. The process of FIG. 11 ends when the three-dimensional object areais set. Otherwise, the process returns to Step S210 to repeat theprocess for another three-dimensional object candidate area.

FIG. 11 shows an example of a three-dimensional object area extractionprocess. Actually, the three-dimensional object areas may be extractedusing another characteristic such as the edge detection result of thevirtual image, or the like, in addition to using similarities of thedensity histograms H (Wi) of the areas as described above.

(Description of Vehicle Accessibility Determination Process)

Next, the operation of the vehicle accessibility determination processis described with reference to FIGS. 12A to 12C. The vehicleaccessibility determination process is performed in the vehicleaccessibility determiner 50 shown in FIG. 1.

Three-dimensional objects on the road surface do not necessarily contactthe road surface at their bottom portions thereof. Specifically, thereare three-dimensional objects each having a floating area which does notcontact the road surface. For example, with regard to the garage 82(three-dimensional object) shown in FIGS. 6A, 6B, only the leg portions83, 85 contact the road surface, but an area between the leg portions83, 85 are floating above the road surface. With regard to the othervehicle 11 (three-dimensional object) shown in FIG. 10, only tirescontact the road surface but the other portion (the body of the vehicle)are floating above the road surface.

Accordingly, in the vehicle accessibility determination process, a roadsurface projecting position, in which the other vehicle 11 is projectedto the road surface from directly above, is calculated. The othervehicle 11 is a three-dimensional object area extracted in thethree-dimensional object area extraction process. This process isperformed in the road surface projecting position calculating portion 50b. The process can detect whether the extracted three-dimensional objectarea includes portions floating above the road surface.

The detection of the floating portions is performed as follows. As shownin FIG. 12A, a line segment L is set. The line segment L extends betweenthe left end and the right end of the other vehicle 11, which is thedetected three-dimensional object area. The other vehicle 11 alsoincludes the road surface grounding positions 94 a, 94 b. Then, thepositions of points, which are above the line segment L and contactingthe other vehicle 11 on the original image 70(t), are detected. As aresult of the process, in the example of FIG. 12A, for example, when asearch is performed from a point Pi, a point Qi which floats in a spaceis detected. Similarly, when the search is performed from a point Pj, apoint Qj which floats in the space is detected. Note that the roadsurface grounding positions 94 a, 94 b are excluded from the object ofthe process since the road surface grounding positions 94 a, 94 b are incontact with the road surface.

Next, the detected points Qi, Qj, . . . which float in the space areprojected inversely on the line segment L to set road surface groundingpoints Ri, Rj, . . . . Note that the road surface grounding positions 94a, 94 b remain as road surface grounding points.

The road surface grounding points Ri, Rj, . . . set as described aboverepresent the road surface projecting positions in which the othervehicle 11 (three-dimensional object) is projected to the road surfacefrom directly above. Among the road surface grounding points Ri, Rjobtained as described above, the road surface grounding points detectedcontinuously in left and right directions are connected to each other tocreate road surface grounding lines L1, L2, L3. The road surfacegrounding lines L1, L2, L3 are equal to one line segment. With thisprocess, it can be understand that the vehicle 10 can move toward theother vehicle 11 until the vehicle 10 reaches at least the positions ofthe road surface grounding lines L1, L2, L3. The vehicle 10 may hit theother vehicle 11 when the vehicle 10 is moved to the other vehicle 11beyond the road surface grounding lines L1, L2, L3.

The road surface grounding lines L1, L2 are connected to the roadsurface grounding position 94 a, and the road surface grounding linesL2, L3 are connected to the road surface grounding position 94 b.Therefore, the road surface grounding positions 94 a, 94 b and the roadsurface grounding lines L1, L2, L3 are unified as one road surfacegrounding line N.

Next, it is determined whether there is a space in the detectedthree-dimensional object area to which the vehicle 10 can access. Thisdetermination is performed in the vehicle accessible space identifyingportion 50 c shown in FIG. 1.

Specifically, for example, as shown in FIG. 12C, it is confirmed wherethree-dimensional object areas are floating above the road surfacegrounding lines Li, Lj, Lk (corresponding to the road surface groundingline N) detected in the original image 70(t).

That is, determination whether the vehicle 10 can farther move beyondthe road surface grounding lines Li, Lj, Lk or not is performed byconfirming that the length of each of the road surface grounding linesLi, Lj, Lk is longer than the width of the vehicle 10, and there arespaces higher than the height of the vehicle 10 above the road surfacegrounding lines Li, Lj, Lk.

Here, the original image 70(t) is generated by receiving a perspectiveconversion. That is, the farther the portions are located, the shorterthe portions are reflected or shown in the upper part of the image.Therefore, actual lengths of the road surface grounding lines Li, Lj, Lkcan be estimated with the vertical positions and the lengths of the roadsurface grounding lines Li, Lj, Lk on the original image 70(t), whichare detected in the original image 70(t). Further, the heights of spacesto which the vehicle 10 is accessible in the positions of the roadsurface grounding lines Li, Lj, Lk can be estimated with the verticalpositions of the road surface grounding lines Li, Lj, Lk on the originalimage 70(t).

For example, as shown in FIG. 12C, when the road surface grounding linesLi, Lj, Lk are detected in the original image 70(t), the heights Hi, Hj,Hk of the spaces above each of the road surface grounding lines Li, Lj,Lk necessary for the vehicle 10 to access beyond each of the roadsurface grounding lines Li, Lj, Lk can be estimated based on thevertical positions of the road surface grounding lines Li, Lj, Lk.

Actual lengths of the road surface grounding lines Li, Lj, Lk and theheights Hi, Hj, Hk of the spaces above the road surface grounding linesLi, Lj, Lk can be respectively estimated based on installation layoutinformation (the height of the camera, the depression angle of thecamera, lens parameters) of the front camera 12 a (FIG. 4).Specifically, since the installation layout of the front camera 12 a isknown in advance, the actual lengths of the respective road surfacegrounding lines L1, L2, L3, and the heights H1, H2, H3 of the spacesabove the road surface grounding lines Li, Lj, Lk can be obtained inadvance by calculation. Then, calculated values are stored in the formof a table in the vehicle accessible space identifying portion 50 c(FIG. 1).

When the road surface grounding lines Li, Lj, Lk are detected, it can bedetermined whether each of the road surface grounding lines Li, Lj, Lkexceeds the width of the vehicle 10, and whether there are spacesexceeding the height of the vehicle 10 above the grounding lines Li, Lj,Lk by referring to the contents of the stored table.

(Description of a Flow of Vehicle Accessibility Determination Process)

Hereinafter, a flow of the vehicle accessibility determination processis described with reference to a flowchart of FIG. 13.

(Step S300)

The road surface projecting positions of the three-dimensional objectareas are calculated as the road surface grounding points Ri, Rj . . . .The specific content of the process is as described above.

(Step S310)

Among the road surface grounding points Ri, Rj . . . , successivelylocated points are unified as the road surface grounding line N. Then,the length of the road surface grounding line N, and the verticalposition of the road surface grounding line N on the original image 70are calculated.

(Step S320)

The height H of the space above the road surface grounding line N iscalculated.

(Step S330)

It is determined whether the vehicle 10 can access beyond the roadsurface grounding line N or not.

(Description of Vehicle Inaccessible Range Display Process)

Next, the operation of the vehicle inaccessible range display process isdescribed with reference to FIGS. 14A, 14B. The vehicle inaccessiblerange display process is performed in the vehicle accessible andinaccessible range display 60 shown in FIG. 1.

FIG. 14A shows an example of a display image 74(t) generated from FIGS.6A, 6B. The display image 74(t) is displayed in the monitor 150 (FIG. 4)of the vehicle 10. As shown in FIG. 14A, in the display image 74(t), theroad surface grounding lines L1, L2 are drawn with bold lines on in thevirtual image 72(t) (overhead view). In addition, in the display image74(t), the three-dimensional object area corresponding to the extractedgarage 82 is superimposed and displayed. Since the vehicle 10 can accessto an area between the road surface grounding lines L1, L2, a bold lineis not drawn in the position of a road surface grounding line. A driverof the vehicle 10 decides that the vehicle 10 can be farther moved tothe rear side of the garage 82 (three-dimensional object) by seeing thedisplay image 74(t). Note that predetermined colors such as red may beadded to the bold lines which indicate the road surface grounding linesL1, L2 to improve visibility.

Note that in the display image 74(t), predetermined gray values (0, forexample) are stored in the invisible areas 86 which are outside thefield of view of the front camera 12 a, and the invisible areas 88 whichare the shadow of the garage 82 (three-dimensional object).

FIG. 14B shows an example of a display image 75(t) generated when theoriginal image 70(t) shown in FIG. 10 is observed. In display image75(t), a bold line which indicates the road surface grounding line N isdrawn below the other vehicle 11. It shows that the vehicle 10 canapproach only to the position of the road surface grounding line N. Inother words, it shows that the vehicle 10 cannot access beyond the roadsurface grounding line N.

It should be noted that the display form of the display image 74 (t) isnot limited to the examples shown in FIGS. 14A, 14B. That is, in theroad surface grounding line N, bold lines may be displayed relative to arange to which the vehicle 10 can access, not to a range to which thevehicle 10 cannot access.

As described above, according to the vehicle accessibility determinationdevice 100 according to the first embodiment of the present invention asconfigured above, the image convertor 30 converts the original image 70including the road surface around the vehicle 10 which is captured bythe front camera 12 a (imager 12) into the virtual image 72 (overheadimage) viewed from a predetermined viewpoint. The three-dimensionalobject detector 40 detects from the virtual image 72 thethree-dimensional object having a height from the road surface. Thevehicle accessibility determiner 50 determines whether the vehicle 10can access to the inside of the detected three-dimensional object or toa clearance among other three-dimensional objects. Accordingly, even ifthe three-dimensional object has a floating area which does not contactthe road surface, it can be detected whether the vehicle 10 can accessto the space or not. Therefore, it is possible to prevent the vehicle 10from hitting the three-dimensional object in advance.

In addition, according to the vehicle accessibility determination device100 according to the first embodiment of the present invention asconfigured above, the three-dimensional object area extracting portion50 a extracts an area corresponding to the three-dimensional object fromthe original image 70. The road surface projecting position calculatingportion 50 b calculates the presence or absence of a floating area thatdoes not contact the road surface and the height of the floating areafrom the road surface relative to the three-dimensional object areaextracted by the three-dimensional object area extracting portion 50 a.The floating area constitutes the three-dimensional object. Also, theroad surface projecting position calculating portion 50 b calculates theroad surface projecting position in which the floating area is projectedto the road surface from directly above. The vehicle accessible spaceidentifying portion 50 c identifies whether there is the space insidethe detected three-dimensional object or in the clearance among otherthree-dimensional objects to which the vehicle 10 can access, based onthe presence or absence of the floating area and the road surfaceprojecting position calculated by the road surface projecting positioncalculating portion 50 b. Accordingly, the presence or absence of thefloating area and the road surface projecting position can be calculatedwith a simple process.

Further, according to the vehicle accessibility determination device 100according to the first embodiment of the present invention as configuredabove, the vehicle accessible and inaccessible range display 60superimposes, to the road surface position of the virtual image 72, thevehicle inaccessible range to which the vehicle 10 cannot accessdetermined in the vehicle accessibility determiner 50, or the vehicleaccessible range to which the vehicle 10 can access determined in thevehicle accessibility determiner 50, and displays the superimposedranges. Accordingly, it is possible to visualize how far the vehicle 10can access to the three-dimensional object even if the three-dimensionalobject has the floating area that does not contact the road surface.Therefore, it is possible to prevent the vehicle 10 from hitting thethree-dimensional object in advance.

Moreover, according to the vehicle accessibility determination device100 according to the first embodiment of the present invention asconfigured above, the three-dimensional object detector 40 deletes thenon three-dimensional objects. The deletion of the non three-dimensionalobjects is made based on a result of the frame difference between thetwo virtual images 72(t−Δt), 72(t) (overhead images) calculated in thefirst frame difference calculator 40 a, and a result of the framedifference between the expected virtual image 72′(t) and the othervirtual image 72(t) calculated in the second frame difference calculator40 b. The two virtual images 72(t−Δt), 72(t) (overhead images) arerespectively generated from the two original images captured atdifferent times. The expected virtual image 72′(t) is expected to begenerated from the original image captured at the same time as the timeat which the original image (t) is captured which is the conversionsource of the other virtual image 72(t), and is expected based on thetraveling amount and the moving direction of the vehicle 10 from thevirtual image 72(t−Δt) which is one of the two virtual images (overheadimages). Also, the three-dimensional object detector 40 detects thethree-dimensional object on the road surface by referring to the edgeinformation of the virtual image 72(t) detected in the edge detector 40c. Accordingly, it is possible to identify the three-dimensional objecthaving a height from the road surface from paints, stains or darts onthe road, or the vehicle shadow 87, and detect the three-dimensionalobject with a simple process.

Furthermore, according to the vehicle accessibility determination device100 according to the first embodiment of the present invention asconfigured above, areas having same shapes (features) and located inclose positions are detected from the first three-dimensional objectcandidate areas detected by the first frame difference calculator 40 aand the second three-dimensional object candidate areas detected by thesecond frame difference calculator 40 b. When the detected areas aredetected to be inclined toward the road surface in a direction away fromthe vehicle 10 based on the edge detection result of the virtual image72(t) by the edge detector 40 c, the areas are detected as areasindicating the three-dimensional objects on the road surface.Accordingly, it is possible to reduce the influence of the timevariation of the exposure characteristics of the camera, shadows,lighting, which may be mixed in the results of the frame differences, byreferring to the edge detection result of the virtual image 72(t).Therefore, the detection performance of the three-dimensional object canbe further improved.

In the first embodiment, an example in which one front camera 12 a isused as the imager 12 is described. However, the number of cameras to beused is not limited to one. That is, it is also possible for the vehicleaccessibility determination device to include a plurality of camerasdirected to the front, the left, the right, and the rear of the vehicle10 so as to be able to monitor the entire circumference of the vehicle10. In this case, the original images captured by the respective camerasare respectively converted into a virtual image (overhead image), andthen combined into one composite image. Processes described in theembodiment are performed on the composite image.

Further, in the first embodiment, an example of determining whether thevehicle 10 can access to the garage 82 or the other vehicle 11, each ofwhich is a three-dimensional object, is described. However, theinvention is not limited to a case where it is determined whether thevehicle 10 can access to the inside of a single three-dimensional objector not. That is, the invention may be applied to a case where it isdetermined whether the vehicle 10 can access to a space between twovehicles to park the vehicle 10 when the other two vehicles are parkedwith the space therebetween in a parking lot which does not have linesindicating a parking space for each vehicle. In this case, accessibilityof the vehicle 10 is determined by detecting each of the two vehicles asthe three-dimensional objects, by respectively calculating the width andthe height of the space between the vehicles, and by comparing thecalculated size (width and height) with that of the vehicle 10.

In addition, the procedure of the image processing described in thefirst embodiment is not limited to one described above in theembodiment. For example, the garage 82 is detected as onethree-dimensional object from the virtual image 72(t) when the garage 82is uninterruptedly reflected within the virtual image 72(t). Assumingabove case as well, a procedure including detecting the road surfacegrounding line N from the virtual image 72(t), and subsequentlycalculating the height H of the space above the road surface groundingline N may be applicable. Taking such the procedure, when the entirethree-dimensional object is reflected in the virtual image 72(t), thethree-dimensional object region extraction process and the accessibilitydetermination process are performed by using only the virtual image72(t). Accordingly, a series of processes can be performed more easily.

Further, according to the vehicle accessibility determination device 100described in the first embodiment, it is configured to obtain the roadsurface grounding line N of the detected three-dimensional object, andto display and inform to a driver only the range to which the vehicle 10cannot access in the road surface grounding line N. However, theinvention is not limited to the above configuration. That is, it may beconfigured to automatically park the vehicle 10 based on the informationof the range to which the vehicle 10 can access in the calculated roadsurface grounding line N, for example.

Moreover, in the first embodiment, the frame difference between thevirtual images is performed to detect the three-dimensional object.However, the frame difference to be performed at that time is notlimited to a frame difference between gray values representingbrightness of the virtual images. That is, it is possible to perform anedge detection on the virtual images and then perform a framedifferences between the virtual images in which the detected edgestrengths are stored. It is also possible to perform a frame differencebetween the virtual images in which the detected edge directions arestored to detect an area where the change is occurred. Further, it isalso possible to divide the virtual image into a plurality of smallblocks and to use similarity of the histogram of the density histogramof each small block and/or the edge detection result.

Although the embodiments of the present invention are described indetail with reference to the drawings, the embodiments are only examplesof the present invention. Therefore, the present invention is notlimited only to the configurations of the embodiments, and it will beappreciated that any design changes and the like that do not depart fromthe gist of the present invention should be included in the invention.

The invention claimed is:
 1. A vehicle accessibility determinationdevice comprising: a memory; and a central processing unit connected tothe memory, the central processing unit being configured to: capture arange including a road surface around a vehicle via an imager to beattached to the vehicle; convert an original image captured by theimager into a virtual image to be viewed from a predetermined viewpoint;detect from the virtual image a three-dimensional object having a heightfrom the road surface; and determine whether the vehicle is capable ofaccessing an inside of the three-dimensional object or a clearance amongother three-dimensional objects, wherein the central processing unit isconfigured to determine whether the vehicle is capable of accessing theinside or the clearance by: extracting an area corresponding to thethree-dimensional object from the original image; calculating a presenceor an absence of a floating area that does not contact the road surfaceand a height of the floating area from the road surface relative to theextracted three-dimensional object area, and calculating a road surfaceprojecting position in which the floating area is projected to the roadsurface from directly above, the floating area constituting thethree-dimensional object area; and identifying whether there is a spaceinside the three-dimensional object or in the clearance among otherthree-dimensional objects to which the vehicle is capable of accessing,based on the presence or the absence of the floating area and thecalculated road surface projecting position.
 2. The vehicleaccessibility determination device according to claim 1, wherein thecentral processing unit is further configured to superimpose to a roadsurface position of the virtual image a vehicle inaccessible range towhich it is determined that the vehicle is not capable of accessing, ora vehicle accessible range to which it is determined that the vehicle iscapable of accessing, and to display the ranges.
 3. The vehicleaccessibility determination device according to claim 1, wherein thecentral processing unit is configured to detect from the virtual imagethe three-dimensional object by: calculating a frame difference betweentwo virtual images respectively generated from two original imagescaptured at different times; converting one virtual image of the twovirtual images into an expected virtual image that is expected to begenerated from an original image captured at time when an originalimage, which is a conversion source of the other virtual image, iscaptured based on a traveling amount and a moving direction of thevehicle for the different times, and calculating a frame differencebetween the expected virtual image and the other virtual image; anddetecting an edge of the one virtual image, and wherein the centralprocessing unit is configured to detect a three-dimensional object onthe road surface by deleting a non three-dimensional object on the roadsurface based on results of the calculating and the detecting.
 4. Avehicle accessibility determination device comprising: a memory; and acentral processing unit connected to the memory, the central processingunit being configured to: capture a range including a road surfacearound a vehicle via an imager to be attached to the vehicle; convert anoriginal image captured by the imager into a virtual image to be viewedfrom a predetermined viewpoint; detect from the virtual image athree-dimensional object having a height from the road surface; anddetermine whether the vehicle is capable of accessing an inside of thethree-dimensional object or a clearance among other three-dimensionalobjects, wherein the central processing unit is further configured todetect from the virtual image the three-dimensional object by:calculating a frame difference between two virtual images respectivelygenerated from two original images captured at different times;converting one virtual image of the two virtual images into an expectedvirtual image that is expected to be generated from an original imagecaptured at time when an original image, which is a conversion source ofthe other virtual image, is captured based on a traveling amount and amoving direction of the vehicle for the different times, and calculatinga frame difference between the expected virtual image and the othervirtual image; and detecting an edge of the one virtual image, andwherein the central processing unit is further configured to detect thethree-dimensional object on the road surface by deleting a nonthree-dimensional object on the road surface based on results of thecalculating and the detecting, wherein the central processing unit isfurther configured to detect: areas from first three-dimensional objectcandidate areas detected by the calculating and second three-dimensionalobject candidate areas detected by the calculating, the areas havingsame features and located in close positions relative to each other; andthe areas as areas representing three-dimensional objects on the roadsurface when the areas are detected to be inclined toward the roadsurface in a direction away from the vehicle based on a result of thedetecting.
 5. A vehicle accessibility determination device comprising: amemory; and a central processing unit connected to the memory, thecentral processing unit being configured to: capture a range including aroad surface around a vehicle via an imager to be attached to thevehicle; convert an original image captured by the imager into a virtualimage to be viewed from a predetermined viewpoint; detect from thevirtual image a three-dimensional object having a height from the roadsurface; and determine whether the vehicle is capable of accessing aninside of the three-dimensional object or a clearance among otherthree-dimensional objects wherein central processing unit is furtherconfigured to: superimpose to a road surface position of the virtualimage a vehicle inaccessible range to which it is determined that thevehicle is not capable of accessing, or a vehicle accessible range towhich it is determined that the vehicle is capable of accessing, anddisplay the ranges, wherein the central processing unit is furtherconfigured to detect from the virtual image the three-dimensional objectby: calculating a frame difference between two virtual imagesrespectively generated from two original images captured at differenttimes; converting one virtual image of the two virtual images into anexpected virtual image that is expected to be generated from an originalimage captured at a time when an original image, which is a conversionsource of the other virtual image, is captured based on a travelingamount and a moving direction of the vehicle for the different times,and calculating a frame difference between the expected virtual imageand the other virtual image; and detecting an edge of the one virtualimage, and wherein the central processing unit is further configured todetect the three-dimensional object on the road surface by deleting anon three-dimensional object on the road surface based on results of thecalculating and the detecting, wherein the central processing unit isfurther configured to detect: areas from first three-dimensional objectcandidate areas detected by the calculating and second three-dimensionalobject candidate areas detected by the calculating, the areas havingsame features and located in close positions relative to each other; andthe areas as areas representing three-dimensional objects on the roadsurface when the areas are detected to be inclined toward the roadsurface in a direction away from the vehicle based on a result of thedetecting.